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ABSTRACT 


Under the constraint of limited testing time, many attempts 
have been made to incorporate prior knowledge and exper- 
ience into a quantitative assessment of reliability. This 
type of technique is known as Bayesian statistics. Since 
the length of time available for testing integrated circuits 
is frequently very limited, an analysis of Bayesian methods 
when applied to the integrated circuit testing problem was 
conducted. 

The critical point of any Bayesian analysis concerns the 
choice and quantification of the prior information. This 
report is a study of the effects of prior data on a Bayesian 
analysis. Comparisons of the Maximum Likelihood estimator, 
the Bayesian estimator and the known failure rate are pre- 
sented. The results of the many simulated trials are then 
analyzed to show the region of criticality for prior information 
being supplied to the Bayesian estimator. In particular, 
effects of prior mean and variance are determined as a 
function of the amount of test data available. 
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1 . 


Introduction 


The lack of integrated circuit reliability data is a chronic problem 
for those concerned with the problem of reliability estimation. The data 
limitations are often manifested in terms not only of quantity but also 
quality. For example, good quality data would not only include all the 
conditions under which the data was obtained but would also include the 
assumptions and restrictions to be used with such data. This needs to be 
considered since quite often data from various sources are not compatible 
with the application the analyst is trying to treat. Data restrictions 
such as these can severely limit the amount of useful data to be applied 
to reliability estimation. 

Faced with situations in which data is not representative or is 
inadequate, it is not uncommon for the reliability analyst to devise a 
rationale which may, in part, be based on "engineering judgment". Unfor- 
tunately, this rationale often tends to be arbitrary and leads to inconsist- 

i 

ency with respect to device types and applications. Therefore, a defined 
rationale was sought which would allow the incorporation of both judgment 
and actual data in a consistent manner. Specifically, judgment must be 
quantified and combined with data. 

There are several rationales for estimating the probability of success 
for a given type of integrated device. The usual basis for derivation of 
such models is the failure-rate-estimate. For these reasons a rationale 
is sought which can estimate integrated circuit failure rates and still 
be compatible with limited data. 

The basic approach taken was to develop a technique which would 
assess integrated circuit parameters, such as failure-rate and life time. 
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This technique of assessing integrated circuit parameters was achieved 
by combining limited statistical data and engineering judgment. A rationale 
was sought which would combine both data and judgment so that an analytical 
estimation of the failure rate for integrated circuit devices could be 
achieved. 

This report emphasizes the selection of appropriate statistical dist- 
ributions for incorporation with Bayesian statistical theory. The resulting 
Bayesian application with various data inputs was then compared with classical 
estimators of failure rate. + 

Since Bayesian Statistics provides a convenient way of incorporating 
prior knowledge or judgment regarding probabilistic events, the treatment of 
the previously stated reliability estimation problem was approached through 
an application of Bayesian statistics. Bayes' Theorem, or Bayes' Rule, can 
be expressed by the formulation given in Equation 1. 

P(AIB) = (I) 

j-i 

The C^, C^, , C n in Equation 1 represent n mutually exclusive 

events. The respective probabilities of these events are denoted by p(C^) 
i = 1, 2, . . ., n. 

The event B is an event for which one knows the conditional probabilities 
P (B jCh). This is read: the probability of event B given event Ch. Once 

these probabilities have been defined, the probability of obtaining a 
particular event C. given event B is given by Equation 1. In the context 
of this reliability estimation problem, Equation 1 must be redefined for the 
continuous case with regard to a specific notation. 


2 


+defined in reference ( 9 ), pages 102-4 
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( 2 ) 


fm fa) 
f Kru') fa')jy 

J A 

the values assumed by the propulation failure rate. 

the event called failure. 

a probability density function for /I . 

a function giving the probability of failure given 
a particular A . 

a probability density function of /| given the failure event F. 

The controversy about Bayes 1 Theorem concerns the uncertainty in knowing 
the probabilities P(Ch) i = 1,2, . . ., n and j? ( /\ ) in the respective 

equations 1 and 2. This problem results from a lack of appropriate data. 
Thus, engineering judgment must be applied in the initial phases of the 
applied Bayesian analysis. The effects of this judgment on P(C^) and f ( A ) 
are lessened by an allowance for incorporation of the appropriate data. In 
this case Bayes' Theorem is applied in an iterative fashion which allows 
the incorporation of data as it becomes available. A better probability 
estimate is obtained as increasing amounts of data are incorporated. 

The initial data will be referred to as prior information. It will 
be defined by a prior formulation or function. Subsequent data will result 
in a posterior formulation. In the case of Equation 2 an iterative tech- 
nique describing this situation is shown below. 


fair) = 


Where: 

X 

F 

Pa) 

&F i b - 
Pair) = 
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Prior information is quantified by r 0 (Ai ^ > t 0 ) given the initial 

information F and time t ; Substitution of this notation into Equation 
o o 

2 yields: 


Where 


fee u) U) 

( KfiaWUa' 


(A ) is the more convenient notation for £ 0 (A I 


is the posterior' function which replaces 
r* (\) and reflects the failure event new F^ (t^ understood) 


With more data, y (A ! F^ may be used to derive another function which 

incorporates this data. Again Equation 2 is used. 

f(tl» ),q-) 
ffeiy) 

£,h) • f,0 1 F, , h) 

Depending upon the quantity of data, this type of iteration can be carried 
out to an arbitrary number of steps. The n^* 1 iteration will take the 
following form. 


Therefore : 


z (Mk) 



N f(T.u) L,a) 

10 1 rj » 

my) ua'Ua' 

\ 

Thus, an incorporation of new data is used to redefine the functional 
description of values associated with a population of particular device 
types. 
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An essential constituent of this Bayesian application is the determination 
of the functional relations which can he used to satisfy the iteration. This 
determination must start with formulations of the prior information. Toward 
this end there will be an effort below to identify the usefulness of given 
distributions for best describing this information; to determine the criticality 
of distribution shape parameters; and to find the influence of varying amounts 
of data. 

The final factor in the approach of this Bayesian application concerns the 
analysis at iteration termination. For this paper, iteration termination will | 
be defined as the point where data will cease to be incorporated into the 
Bayesian formulations. This termination point will be determined through a com- 
parison of Bayesian failure-rate-estimates with classical failure-rate-estimates 
for equivalent data. 
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2 . 


DISTRIBUTION FUNCTION STUDY 


Before the details of the Bayesian investigation can be pursued, the 
functional relations presented in Equation 2 must be defined. 

The probability of failure given a particular failure-rate was denoted 
by j* ( T" | A ) . It will be assumed that the exponential distribution for 
times-to-failure holds. Because of the relation between the exponential and 
Poisson distributions, r (F I \ ) can be expressed in the following Poisson 

form: 



where F = the number of failures. 


( 6 ) 


The interpretation of this formula is that, for failures occurring at a 
given rate \ , the probability of having exactly F failures in time t is given 
by f(F \ V j ( i*e. ; Equation 6) . 

The lambda ( ^ ) given in equation (2) can be thought of as the 

constant rate of failure for an exponential distribution of times-to-failure. 
Furthermore, the density function j~ 0) represents the prior estimate 
of the failure rate for a given integrated circuit device type. This density 
function can be thought of as describing the prior probability that ^ is 
one of the values between zero and infinity. This probability is a theoretical 
description of the distribution for the entire population (not sample) of 
integrated circuits of a particular type. 

n , . . 

The selection of the formulation for j. ( ^ ) in Equation 2 will have to 
be subject to some fundamental selection criteria. This is because all 
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distributions selected for f(,4 ) may not be amenable to the iteration 
process already discussed. As was stated before, the iteration process 
lessens the initial uncertainty by building the true shape for the function 
f ( X ) as data becomes available. 

Because of this uncertainty, the function selected must be viable — 
this means that incorporation of data through iteration is not only per- 
ceptible but meaningful. This criterion can be thought of as a sensitivity 
of shape variability due to the iteration process. The second criterion 
sought regards the result of Equation. 2 ; i.e. , f ( ^ IF - ). For convenience in 
the iteration process, the result of ■ Equation 2 must be amenable to reuse 
as f( X ) in the next successive stage of computation. A closed function 
form is sought for ease of computation. 

The first functions considered were of standard form and can be found in 
basic statistical texts (Ref. 5). The formulations and conditions for these 
functions are stated in the following section. 
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3. DISTRIBUTION SELECTION 


If one considers the criteria previously discussed and the functional 
conditions, a likely candidate for this Bayesian application can be selected. 
Table I provides some remarks necessary for this determination. 


TABLE I 


NAME 


Uniform 


Exponential 

Gamma 

Normal 

Beta 

Cauchy 

F-Distribution 
Chi-Square 
Inverted Gamma 

Converse Inverted 
Gaimma 


Function does not yield closed form when iterated 
- computation difficult. Not defined on the entire 
(o, °o ) domain 

Iteration yields a gamma form 

Iteration yields a gamma form - see Section 4. 

Computationally difficult when iterating 

Only defined for the domain (0,1) 

Moments do not exist therefore no expected value 
can be calculated. 

Only defined for the domain (0,l) 

Special case of Gamma 

Does not describe the correct random variable 
describes the estimate given the true. 

Parameter definitions difficult to make resulting, 
in computational difficulty. 
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PROBABILITY DENSITY FUNCTIONS 


Equation Name F ormulations Conditions 


(7) Uniform f(x) =j 

. <*L < x < 

/£ - oc ^ 

c O Elsewhere 

(8) Exponential f(x) =, 

r 

1 V 9 g-^ X //S ) X > 0 9 > 0 

(^O Elsewhere 

(9) Gamma f(x) = 

flCX ^' 1 £ ) x > o <* > o, > 0 

) O x < 0 

K such that / f(x) dx = 1 

- -aO 

K = 3 ^ 

(10) Normal ffx) = K \ - cO < x < + ©o , <A > o 

' ' ' ; V ^ 1 ' 

(11) Beta f(x) =J 

K ^ _1 (1-X)^ -1 0 < x < 1 

O Elsewhere 

(12) Cauchy f(x) = 2 2 - ©0 < x < + 00 

a + x 

(13) F-Distri- f(x) = , 

bution 

f KX K /2 )-l K / 2,-1 

(l-x) 0 < x < 1 

l O Elsewhere 


x K-2) / 2 p -(x/ 2 ) 

(14) Chi-Square f(x) = f - _ 

L /2 ri^/2) 


o 
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There are two further functions which can be derived for this Bayesian 
application. It was derived from basic statistical and reliability theory. 
The results of this derivation are stated below. 


F unction 

(15) 


Name 

Inverted 

Gamma 


Formulation 


Conditions 


f(x) = 


(>*-) 


- (V x t +1 ^ °- 


O 


Elsewhere 


(16) 


C onverse 

Inverted 

Gamma 


f(>| x) 


Ukt ^ 

I (o< + /J_) yA- > o 




V. 


> O 
$ > O 

> o 

Elsewhere 


Equation 15 is derived via the standard change of variable technique. 

The basic function treated was a density function for the random variable 
associated with times -to-failure for a given number of failures ( A* ) given 
their rate of occurrence ( X ) * Equation 15 represents the distribution 
of the random variable called a failure-rate-estimator. 

Equation l6 is a description of the random variable called true-failure- 
rate given the failure -rate -estimator information. 

The derivations of formulations 15 and 16 are contained in Appendix A. 

It will also be noted that, after simplification, these derivations yield the 
same result as the formulations presented in Section 4. The formulations of 
Section 4 are, however, more easily derived and understood. 


Q 
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The gamma compliance to the defined selection criterion can he demonstrated 
by the formulations below. These formulations are used in Section 4 to 
illustrate the Bayesian iteration technique and parameter determination for 
the general gamma formulation. Complete derivations are deferred to Appendix B. 
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4 . PROPOSED BAYESIAN ITERATION TECHNIQUE 


Substituting Equations 6 and 9 into Equation 2 yields the following 
formulation: 


f ( A I F) = (K/F l) e" At (AD F (17) 




(v * N - / t , J F -1 - ( A /£ ) / 

C k / f ! ) e (At) ^ e 


where K = 


A= o 

l 


^ P M 


F = the number of failures in time t 
The derivation for K is given in Ref. ( 5 ). 

) ( ot ) is the gamma function 


12 
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Using the notation outlined for iteration in Section 1, prior information 


is factored into the following equation: 


< A I F c ) = K o X 


-1 


e 


- ^ 1/6 


(18) 


where K = 
o 


r<*> 


K is determined by the initial constants ©C and/^. 
o 


With the above prior information and iteration algorithm, the following 
sequence may be derived. The successive steps are presented below, however 
the derivations appear in Appendix B. It is important to note that K 
( i = 1, 2, 3? • • n) is being used as the number of failures occurring in 
t ime t^. Since the times-to-failure (t^) are being generated one at a time, 
is always unity (i.e., F^=l, i - 1, 2, . . n ). 

^UK). k, + F i>-> 

where K[ = ^ ( l /^ + tj)** + ^ j /] (ct + Fj) 


(19) 


2 (MF 2 ) = A 


U+f 1 + f 2 ) -l -A( *jjs +t 1 + t 2 ) 


where K 2 = + tj + t 2 ) 


* + F i + F 2 


r 


(dv+ F l + F 2 ) 


( 20 ) 


m 




w cx + j; F; 

where t.) 


- M 1 h + it.) 

(2 



(21) 
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It should be noted that iteration of Equation 18 through 20 produces 
successive gamma formulations. 

4.1 INFORMATION DERIVED FROM BAIESIAN TECHNIQUE 

The information derived from the Bayesian technique will come from 
the probability density yielded by Equation 2. The standard methods of 
treating the problem of finding an average value for a random variable 
given its density function is via the theory of mathematical expectation 
(Ref. 5 ). The expection of /\ is defined by the following formula: 

t 

E (\) = }■(■(/) dA' o< A £ (22) 

\ > 

where A = the random variable representing failure 
rate for a device population. 

p ( A ) - fc he probability density function for \ • 

Equation 22 represents the mean for a particular density function 

n 

- ( A )• A faction which is related to (22) is the formulation for the 
variance and deviation for the density function ^ (^\ ). These are 
respectively defined with the following notation. 

E [(A- E (A) ) 2 ] = VAR (A) = (A'-E(A')) 2 f(A') dA" ( 23 ) 

. A P s 

The standard deviation for the density function JL (/\) is. given by 
the square root of Equation 23 . 

As regards this particular application, the mean and variance for £( A ) 
are derived in Appendix C. For convenience, the respective formulations are 
stated on the following page. 


14 
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n 


e (A) = o< + y f. 

n Z_ i 

i=l 


H 


n 


v + L h 

i=l 


(24) 


E n [{ A - E( A ) ) 2 ] 


= * + t, F i 

i=l 

ri/ + n 
'* X- 'i 

i=l 


(25) 


Where n = the number of Bayesian iterations 
K = 1 (i.e., ITT; = ^n) 

The above equations are read as the mean and variance after n iterations 


4.2 INCORPORATION OF PRIOR INFORMATION 

As was stated previously prior information can be derived either from 
experience or from "engineering judgment". A problem with both experience 
and judgment is the quantification of these sources. It should be noted 
from Equation l8 that the functional relation for j|_ ^ | F q ) is use- 

able only if the gamma constants oL and are determined. 

Consider the case where prior data can be utilized in determining 
the constants oi , jg Suppose that data is available which indicates 

that the mean and variance of failure-rate-estimators for a particular 
device are M and V respectively, theR the parameters oi , ^ of the given 
distribution may be computed via a system of simultaneous equations. The 
following equations are for the mean and variance of the gamma distribution 
of Equation 18. 
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E( b ) = */S 


(26) 


VAE ( A ) = oLfi 2 (27) 

When Equations 26 and 27 are set equal to M and V respectively, 

c< = M 2 (28) 

V 

/? = V (29) 

M 

If engineering judgment is the only method available for assessing 
prior information, quantification is similar to the above. For this 

case, the values for M and V may be judiciously chosen. Insertion into 
Equations 28 and 29 will yield the appropriate distribution constants. 

The consequences of mean and variance choices will be deferred to Section 5. 

Once the constants and have been chosen, one may proceed with the 
iteration algorithm already defined. Any type of data can be used for this 
iteration providing failures and times can be specified. There is only one 
caution with regard to the data used. Data entered into each successive 
iteration must be derived from similar device lots and application. 


16 
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4 . 3 CONVERGENCE PROPERTIES 


Evaluation of this Bayesian technique will be conducted by comparing 
it with the classical maximum- likelihood-estimator for failure rate in 
Equation 9* It can be shown that the two estimators, Bayesian (Equation 24) 
and Classical converge as the amount of applied data increases. This com- 
parison may seem to be a rather trivial aspect; however, it is important. 

The convergence shows that the prior bias supplied to the Bayesian is not only 
removed but converges to the true rate with the incorporation of sufficienct 
data . + 

There is a further convergence property which is of interest. It can 
be shown that, as the prior variance of the Bayesian becomes large, the 
Bayesian and classical estimators converge. This convergence is independent 
of the amount of data incorporated. To see this consider the following argument. 


Prior Mean = M = d ^ 

; 2 

Prior Variance =V = <*£ 
P ^ 

From these equations, 

^ = V 

' J? 

M 


(30) 


This implies that O//# is inversely proportional to V for a given M . 


Also from Equations 30, 


M 


oi = 


This implies that <X 


is inversely proportional to for a given M . 


+ This formula is true provided that the prior density is not zero at the 
true value. While this is not a problem with the gamma distribution, it 
warrants special attention if a discrete prior is used (see Appendix D). 
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Consider the Bayesian expected value 


(24) 


m - 


c>( ■*- 21 ^ 


It has been shown that as V increases with fixed , ex' and 

P 

decrease (the limit for both being zero). Therefore E(^ ) approaches 
in the limit. This ratio is the classical maximum- likelihood-estimator . 



18 
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5. 


SIMULATION TECHNIQUE 


Because of the difficulties and unreliability in actual failure data 
collection, failure simulation was used as a workable solution. 

The method of this computer simulation was to use a random-number generator 
to generate exponentially-distributed times -to-fai lure. This method can be 
thought of as a way of collecting "perfect" data. 

Failure-time simulation is accomplished when values of F(t) are supplied 
for a given rate . 

The simulation generates failures and failure-times based on a failure 

rate The results, allow computation of a failure-rate-estimator , 

which approximates the true rate, ^ *, and can be compared to it. Simulation 

also provides an ideal way in which both Bayesian and classical estimations 

of failure-rate can be compared while data are incorporated. 

This comparison is done for “the purpose of checking Bayesian and classical 

changes with respe.ct to data and convergence properties. The simulated data 

also allows for the performance (with respect to time and failures) of the 

Bayesian and classical estimators to be studied given any amount to simulated 

data. Performance will be defined as an estimator's nearness to the true rate 

( As an illustration , suppose that a given data base is supplied and that 

estimate £ f , and estimate <£ z are calculated. The estimate which is nearest 
* 

to after the incorporation of the same data is said to perform better. 

Given a specific prior mean (M ) an( i prior variance ( V^) , it has been 
shown above (Section 4.2) how simulated failures and their corresponding times 
are incorporated into the Bayesian formulation (See also Appendix B below'). 

The respective estimators can now be compared to each other and also to ^ # 

This can be done data point by data point. An example of this can be seen in 
Figure 1 which represents the incorporation of 50 simulated data points. 
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Since this simulated data is generated in a pseudorandom fashion, 


one simulation run may yield a different comparison chart than another. 
Variation in the random-number-starts causes this situation even though 
prior information (M^, V^) remain constant. An example of this difference 
is shown in comparison of Figure 1 to Figure 2. 

To average out the effects of these random-number starts on the data 
to be analyzed, comparison information (as Figures 1 and 2) will be generated 
a given number of times. In the examples which follow, 500 trials were 
computed then compared for performance. The results were then normalized to 
find the percent of time for which the respective estimators performed better. 

Given M and V , suppose that the Bayesian and classical estimators are 
to be compared with respect to A . This comparison will be done after 
the incorporation of the 50.th data point for each of the 500 trials. Thus, 
a measure of the averaged performance of the respective estimators can be 
obtained. 


This measure may be defined in the following manner. 


Bayesian: Measurej = Jt 5o 0) - A | i=l, 2,3? . . . 500 trials 

Classical: Measurej = £.^T So (\) - \ | d=l,2,3* • • • 500 trials 

where: EST ( A ) is the classical failure rate estimator 
after fifty data points are incorporated. 

As each of these 500 measures is computed, they can be compared and 

counted. A simple counting routine will keep track of the percentage of the 

500 trials for which each of the measures is best. Since the number of trials 

is large, this percentage statement is, essentially, a probability-of-occurrence . 


20 
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The previous discussion was conducted under the assumption that M 

P 

and V remain fixed. If were allowed to vary in such a way as to bracket 
then the portion of time for which measure i is less then «j can he plotted vs M 

I 

A Bayesian performance distribution is thus achieved. This distribution would, 
in effect, show how Bayesian performance is affected by errors in the selection 
of the prior mean with respect to \ . 


In order to have V and M follow the same relation while M is changed, 

P P P 

a simple formulation was derived. A relation was chosen between V and M 

P P 

such that V would have a fixed relation to M while M is varied with respect 
P P P ■ 

to 


y. 


Suppose 


M 

K = D 2 " 


(32) 


Where D = prior standard deviation + 

ir 


K = arbitrary constant 


+ The spread or dispersion of a given distribution (in this case, of gamma 
gamma form) is specified by the variance of that distribution. The 
positive square root ( ) is defined as the standard deviation. 
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The standard deviation was used in this ratio since it is usually 
closer in magnitude to the mean than the variance and therefore is more 
easily visualized. 


Solving for D yields: 

ir 


D = M 

P _E 

K 


By definition, the corresponding variance (V ) is: 

P 

2 


V = 
P 


M Y 
P) 


K 


Thus, for a given K, the fixed relation between the prior mean and prior 
variance is established. 


Figures 1 and 2 represented a large simulated data base of fifty points. 
Since an objective of this report is to treat the situation where only limited 
data is available, the previous procedure will have to be restricted. To 
represent this situation, simulated data was generated and the measures i, j 
were applied at the tenth data point. This was an arbitrary selection since 
other points could have been chosen. 

Simulation was used to determine how the Bayesian performance distribution 
was affected by various K factors. The true rate ( \ ) was fixed so that the 

effects on Bayesian performance with respective wide and narrow variances could 
be observed. 

Figures 3-H demonstrate the effect of a wide variance incremented toward 

a narrow variance. The constant K of Equation 32 increases with each figure. 

-6 

These figures are computed with = .05X10 failures/hour. For the purpose 

of this analysis ^ * was kept at the rate specified. The same effects could be 
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observed for other A values. However , the range of K factors 

may change somewhat in order to produce the same demonstrated effects. 

Each figure represents Bayesian performance over a range of prior 

mean (M ) values. Only the K factor was changed in each figure. Each point 
P 

depicted in the figures was computed from ^00 trials. 

Analysis of Figures 3-il will be deferred to Section 6.2. 
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6 . 


RESULTS 


6.1 Analytical Results 

The analytical results which were derived from this study may be 
summarized as follows: 

1. Computationally ? incorporation of data is independent of unit size. 

There is no difference between one computation containing n failures 
and n computations considering n successive failure times. However , 
trend indications and evaluations are facilitated by making com- 
putations at the smallest unit available. 

2. In the limit, as the amount of data becomes large, Bayesian 
and classical estimators converge. (Section 4.3) 

3. For a given amount of data, a great increase in the Bayesian 
prior variance implies that the Bayesian and classical estimators 
give the same results. (Section 4.3) 

4. The prior parameters ^ of the assumed gamma density can be 
computed form easily understood statistical mean (M ) and variance 
(V ) statements (Section 4.2) 

6.2 Analysis of Observations Generated from Simulated data. 

Until now it has been understood that the Bayesian technique is pre- 
dicated upon the selection of a prior mean and variance (Mp*V ) for the 
gamma density. These values are arrived at using some prior belief and/or 

data. However, the qualitative relation between M and V must be understood 

p p 

before the Bayesian technique can be discussed further. What does this 

relation between M and V mean? 

P P 
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In the simplest terms the value of V expresses the confidence one has 


in the value selected for M . This is because the variance, by definition, 

P 

is a measure of the spread or dispersion of the selected prior gamma density, 

A narrow variance implies high confidence in (ie., you believe / is at, 

or near to, M ). A wide variance implies that one is less sure about the 

\* + 
position of with respect to M^. 

Related to this variance/confidence relation is the impact of V upon 
successive Bayesian estimators (ie, incorporation of data). It has been 
shown in Section 4,3 that as increases, E( ^ ) approaches the classical 
maximum- likelihood-estimator • One would expect to see fluctuation in the 
initial classical estimator, because of the random inputs. One would 
likewise see a fluctuating E( ^ ) for a sufficiently wide V . This relation 
can be demonstrated by Figure 12. (Note that when only one plot character 
appears, ordinate equality is indicated.) 

Empirical evidence shows that as becomes narrow, indicating confidence 
in M , the Bayesian estimation sequence has much more inertia with respect to 
data inputs. This means that the sequence is much more stable. As a consequence, 

i* 

large quantities of data are required to achieve good estimation of A should 
Mp be largely in error from A • 

However, if M ^ )\ , the stability aspect is an obvious asset to the 

P 

Bayesian estimation technique. These observations are demonstrated in 
Figures 13 and l4 respectively. 


+ It must be understood that A is known only for simulation purposes and 
analysis .• Actual testing does not allow this to be the case. 
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Since in reality one can not be sure where M is with respect to ^ , 

,* P 

it is obvious that M , with respect to X and selected V values must be 

P r P 

investigated. The Bayesian performance-distribution is a means of invest- 
igating these features. Bayesian performance-distributions were defined 
and discussed in Section 5- The results of these empirical studies is 
discussed below. 

The major questions to be answered by this analysis will be concentrated in 
two areas. 

1. What is the effect on the Bayesian performance distribution when 
Vp changes? 

2. What is the effect on the Bayesian performance distribution when 

M is in error with respect to \ ? 

P 

In addition to the relation between: M and V previously discussed, V 

P P P 

also has definite effects upon the Bayesian performance-distributions. In 

order to examine this effect, three items were fixed. First, the simulated 

true failure-rate remained fixed at .05 failures/10^ hours. The range of 

prior mean values was fixed at £.012, .093 failures/10^ hours. The relation 

defined between M and V also remained constant (Equation 32). For each of 
P P 

the successive figures (3,11)* the prior variance was allowed to become 
narrower by increasing the constant K or Equation 32. 

The following values will be used in the discussion below. 


Lower distribution tail = < .04 failures/10 hours 

Central distribution region = [.04, .0623 failures/10^ hours 

! g 

Upper distribution tail = > .062 failures/10 hours 


It is apparent (Figure 3) that, for a sufficiently wide V (k= .0006), 

Jr 

the performance of Bayesian estimation over classical is equivalent to a 

/ 

coin toss. This holds for the range of M given 10 data points for each 

P * 

simulation. 
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The remainder of the distribution figures (4-11) are summarized in the 


following table. 
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TABLE II 


K FACTOR EFFECT ON BAYESIAN PERFORMANCE DISTRIBUTION 

.0006 ^ K < .003 . tail probabilities remain relatively constant 

. central region probabilities increase 


.003 ^ K ^ .3 • central region and tail probabilities remain 

relatively constant 


K > -3 


. tail probabilities decline first 
. central region probabilities decline next 
. final phase leaves spike which coincides with A 


There is a particularly significant feature of Table II. This feature 
concerns the "K" factor interval £.003, .3] where the Bayesian performance 

distribution remains relatively stable for the prior mean ( ^ + .04) X10 

This , stability is not affected by the value of dictated by the K factor. 

This implies that there is a range of prior values (M^, V^) which produce optimum 
performance of the Bayesian technique. The lack of distribution uniqueness for 
the above K interval is important since we are primarily concerned with limited 
prior knowledge and/or data. This stability implies that one does not have 

to be super -critical about the selection of M or Y . 

P P 

The following fact should.be noted. The true or simulated rate was 
specified to be .05 failures/lO^ hours. Other studies indicate that the 
results generated from other simulation rate values would be similar, but not 
necessarily the same, quantitatively. 
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Analysis regarding the criticality of choosing particular values 


for the prior Bayesian formulation will be treated below. 

The experiments used to generate Figures 3-11 were done for the purpose 

of graphically representing the probability distribution of Bayesian failure- 

rate-estimators. This was done on the domain of prior mean values, M = 

,* . g 

\ + .04 failures/10 hours for fixed K. 

Given a pre-selected prior mean (M ), criticality of selecting a prior 

value of K from Equation 32 will be determined. If the Bayesian technique is 

to be used to the best advantage, K (given M ) should be selected such that 

B 

the highest probability of getting a better Bayesian estimate (ie., closer to 
) relative to the classical is obtained. 

It can be observed, from Figures 3-11? that a uniquely optimum K value 

relating to the above criterion does not exist. It appears that the highest 

probabilities are obtained when K is in the interval £.003, .3] (see Figures 

6-8). This means that, for a given M , K in the above interval can be selected 

without essentially lowering the probability of obtaining a better Bayesian 

estimate (ie, closer to A ) relative to the classical. From Equation 32, 

it can be shown that the corresponding values of prior deviation (D ) resulting 

3? 

from the end points of the K interval are given as follows: 


3.33 (M ) -S D 


P 


333.33 (M ) 


One can .therefore expect good Bayesian performance with respect to 
classical without severe restrictions placed upon either prior mean or variance. 
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6.3 SUMMARY 

1. Appropriate functions and distributions were selected for the 
application of the continuous Bayesian technique. 

2. Treatment of a discrete form of the Bayesian technique was analyzed 
and is presented in Appendix D. 

3. An easily computed iteration technique was presented. This allows 
easy incorporation of new data as it becomes available. 

4. Simulation results and graphics were presented in hopes of 
facilitating Bayesian understanding. 

5. The criteria are defined under which the Bayesian technique is 
superior to classical estimation 'techniques . 

6. It was shown that severe restrictions on prior Bayesian information 
are not necessary. 


7.0 REFERENCES 

l) Babillis, R. A., Smith, A.M., "Application of Bayesian Statistics 

in Reliability Measurement", General Eleqtric Technical Information 
Series, No. 65SD250, April 1965 • 


30 


JTPL Technical Memorandum 33-614 



7.0 REFERENCES (cont'd) 


2) Breen, T. J., Koehler, R. D., "implementation of Bayesian . 
Techniques at GE - RDS", General Electric Technical Information 
Series on Bayes Theorem and Its Applications to Reliability 
Evaluation, No. 67SD213, April 1967. 

3) Canavos, George C. "A Bayesian Approach to Parameter and 
Reliability Estimation in the Poisson Distribution", 

IEEE Transactions on Reliability V.R-21, No. 1, February 1972, 
pp. 52-55. 

4) Easterling, Robert G., Weeks, David L. , "An Accuracy Criterion 
for Bayesian Tolerance Intervals", Royal Statistical Society 
Journal, No. 2, pp. 236-240, 1970 

5) Fruend, J. E., Mathematical Statistics , Prentice-Hall, Inc., 1965. 

6) Hoel, Paul G., Introduction to Mathematical Statistics , John Wiley 
and Sons, Inc. 1966. 

7) Parzen, Emanuel, Modern Probability Theory and Its Applications, 
John Wiley and Sons, Inc. 1962. 

8) SimKins, D. J., "Weibull, Bayes and Entropy", Proceedings, 1972 
Annual Reliability and Maintainability Symposium, January 1972, 

San Francisco, pp. 190-199 

9) Van Alven, William H., Ed. ARINC Research Corporation, Reliability 
Engineering, Prentice-Hall, Inc., 1965* 


JPL Technical Memorandum 33-614 


31 



PRIOR VARIANCE: . A9D204095350-015 

PRIOR DEVIATION: . 221*35531853-007 







PRIOR VARIANCE^ , A90204095350-01 5 
PRIOR DEVIATION: . 221H35531853-007 
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Fig. 2. True failure rate, classical estimator, and Bayesian estimator with respect to number of 
simulated trials: \* = 0. 05 X 10“ , V = 0.49 * 10"^, D = 0. 22 X 10 “^ failures/hour 
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stimato 



Simulation Rate = 0.05x10’ Failures 
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Fig. 5. Probability of Bayesian estimator being better than classical estimator, K = 0. 001 
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Fig. 6. Probability of Bayesian estimator being better than classical estimator, K - 0. 003 
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PRIOR VARIANCE: .14 15109122 91-01 & 

PRIOR D EVIAT ION: .3/11793 61362-008 
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True failure rate, classical estimator, and Bayesian estimator with respect to number of 
simulated trials: = 0. 05 X 10"^, V = 0, 14 X 10"^, D = 0. 37 X 10"® failures/hour 

7 T\ 9 XT ' 


PRIOR VARIANCE: . 983*38190^1-015 

PRIOR DEVIATION: .990155639635t008 
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APPENDIX A 


DERIVATION OF FAILURE -RATE DENSITY FUNCTION 


Define a failure-rate-estimator ( A ) as a function of times-to-failure 
( i . e . cumulative ) 


/\ = h (t ' ) 

where h (t 1 ) = r/t f 
( A“ 1 ) therefore /\ = r/t ! 

where t 1 = 21 i 
i*M 

and r = total number of failures occurring at rate^A , is the 
unknown true failure-rate* 


From Reference 7? a relation between sums of exponentially distributed random 
variables and t! r ,t failures occurring at rate " yU " is derived. The resulting 
probability density function is: 


(A-2) 




) 


f > <t‘) & 

rvo 


t'*o 


o ¥ < O 

A. 

This is the density for times-to-failure ( 21 "V; ) to rth event 

i=i 

occurring at rate A* . 


Next, we find, the related density function for yk using Equation A-2 and a 
change of variable. 

The density function for \ is denoted by g( /l ) 
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(A-3) 


v- u-r £*'' 


but 

(A-4) g( A ) 

this is an inverted gamma density. 


*'-f 


so 


(^/u') 7V / | 

R-) U 


A-«- j 


<*r) 


Proof that g( \ ) is a density function 


Let 


X 


j 

J nfe 



Substituting 


x = 


\ 

A 


x 


I 


A e d X 




©O 
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<50 



therefore 

A- 

j _ (alaS) _ , 

ri-> c^r 

However, the converse situation is what we wish to examine. 

That is to say, we wish to have an expectation of the value of the true 
rate {/x) given the estimator ( A ) and the corresponding failures ( r ). 
The derivations which follow are directed at this converse situation. 

A gamma density assumption will be used for the description of the 
density function for . It will be shown that this approach yields 
the same result as the assumptions and derivations of section 3. 
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DERIVATION OF CONVERSE INVERTED GAMMA FUNCTION 




A- yt+v v ^ / r 

cS M o -m U 0- A - T 


no 

Assume that the true failure-rate (denoted by/*- ) has a prior 
gamma density with parameters <X , ^ See Equation A-5 

(a-5) l^)-£ 

I7o 

Using Bayes rule, the converse function is given hy Equation A -6 


(A-6) 


1 ^ , /u ^ =. rK) • ^ 0 1/<, aJ) 

«o 

!-» • ^0 O i 




(5J k e. a- e_ ^ *K) * ^ | /&-, -1) = A- ) Cp /A. 

Ro "T 77 
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Then 


h (.A I ^ = 

Let -- + 

4 

h(^l A vO ? 

li U, 

K C/t I A , a.) 
h (,a I A, / 


•n_ i » 

y- q> 




jr n-i & 


t<X 4- 

^5c G 


n~ .oc 

A $> 


roO 


/■ ' r^ Go 

x)^ 

M A 


A 


(o< -t- -’"L ) - I V X\ )/**■ | 

t, (2 d 


A* - o 


A 


in the integral. Then 


) “A 

(a.VH -GP + t)>- 

^ <S 


y 


~ u U + '0-» 

4 




V 


1 a 


(f + >■ J 


(* + “ 1 



>^-i 

yA- ^ ' 

,oo 


W> * X) 


f U+-^V -H I 

1 H <s ^ ^ 


* 


= 0 


(,<*+A.)-| 

<5 


-[$* %\ 


)A 


&+*) /($ + %) 


+■ A* 


j)= (fA 

f7^U4"Ol.^ 


A *“ . -, Jfi'X)* 

(^A( + A_) CS 


(converse inverted Gamma density) 
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PROOF THAT 


h (yU j A,*J IS DENSITY FUNCTION 


CTO 


( 

oi ^ 

j A; J/^ - ( 



/Uz o 


Let 


+ <*.') ^z. 


((' * %)/<- 

(<£ + %*) <|a- 


( )- * '(^ * I 

L* € 4 " 


h(/^l A,*u) r ((' v /x) 


X •>-'»- 




fZ- + 


(G * %) 




Ctv »- ->0) - ' _ o 

l <) 


cX+ ^ 


% ~) fl«. + 


H* v ao 


a. 


4- *v 


JPL Technical Memorandum 33-614 



EXPECTED VALUE OF THE CONVERSE INVERTED GAMMA DENSITY 


, v. ' (^O-' 

= K y^- 


<3 


where K = 




o( \ rt 


rv + -v) 

= Ia 


-0)^)a 




k \y* n -& 2 


/U-O 

Integrating by parts 




E» = X 


CK At Xl — 


«*x ; 




-($ + X Vs 

e d 


3^ 


Let = (0 * % )y 

V^*?0^ 


3 A- 


E» * K 


oi ±n. 




a 


(<X±n.')-l _ -M | 


3- 


E ^ - C<P* '>U 




- 


c* v 


«♦ x 


Rot v 


(Q+ •%) 


*+ *l + i 


expected value of copverse inverted 
gamma density function 
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APPENDIX B 


GAMMA ITERATION DERIVATIONS 


From the text, it is seen that the general formulation of the 
Bayesian application is given by Equation 17. This equation is 
reproduced here for convenience: 




k/r <s" At uo r 


<x-i 

A <2 




K/pi 


X 




Suppose that initial or prior information is represented by failur 
F , and time t Q . The gamma form of this prior information 
becomes : 


k. 



where 



1 


Suppose that new data of the form Fp t^ becomes available. A 
substitution of this data into Equation 17 yields: 



JPL» Technical Memorandum 33-614 



Simplifying, 


[ (\ I r . 



(B-l) 


The denominator of Equation B-l can be interpreted conveniently using 
a change of variable. 

Let y = ( Zp + ) 

then dy = ( ^5 +• b, ) 

Making this substitution, Equation B-l becomes: 


[GIf, ,0 ■ 



(B-2) 


The integral left in the denominator is the gamma function. This 
integral can be represented by the following notation: 


oO 
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With the appropriate substitutions and simplifications. Equation B-2 


(i. e. , Equation 19 of the text) becomes: 



k, \ <2 


where 




r.) 


Successive applications of the preceding rationale yield Equations 
20 and 21 of the text. 
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APPENDIX C 


Derivations for mean, variance and deviation of the Gamma Function 
given the definition (Equation 22 are presented below): 

Mean Derivations 


E o') = j \\C\) X \ 

Substitute Equation 18 
then 

C \ * 


(c-l) •* > < «o 


Va 


EG) ■ U. 1 A « ’ J) 

A 


where K = 
o 


1 


Integrating by parts yields: 


~ -XA r 

J) 


t(A) * k. \ A’~" o 

1 

L 


Changing variable, let y = / $ 


- Va 


56 


then 


cO 


e tt) = k. <*/£ 


y + ! 


^ I 5 

h- o 


ol - 1 _ - 'I 


e’ * J' 
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With the incorporation of new data F, and t, f ( A ) becomes 
f ( | Fj, t^) as noted in Appendix B. 

Using the Equation 19 and the definition of expected value, 
we get: 



' t.) 

<S iA 
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Integrating, 


E.lO - V* v 


a ^ v\ 


(*+ F, ) - ( -A Ofa + ^ 0 I , 

y <s JA 


y 


A* 


Changing variable i. e. , y = ( Va + ^\) 


e,co= k, 


a* r, 1 

, , ' \ + ft 

X + (i? * ^ 1 ) 


<pO 

u^-v* 1 




U: 


1 


But 

oO 


h * 
/ 

r 




therefore 


Eft) = 


ot -*- T7 

'U +t * 


Successive applications and an induction argument using 
the notation developed will yield Equation 24 found in the text. 


E m Q) - _ ll F; 


£ + 


Li; 


= 1 
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VARIANCE DERIVATIONS 


The variance for a continuous density function f ( ^ ) is given by: 

e[(A - EttV) 1 ] * /(A - r(») 2 ffiO 1A 

i 

Let ^ ^ 


then 


Eia>oi • |fr^‘ fa) 

A 


r W\ ,Y-) 

C - /«•; j 


(f\\ \ 

f 'A j O 


“ tk. 


Oi- I 


From Equation 18, f( \ ) = K q \ <g 


Vfi 


where 


K 


3* 17.0 


Therefore 




E lu ->?] - k. \ 
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V 

In changing variable, let M r S,<5 

0 

Jr t 

Integrating twice (by parts) yields the following equation. 


ELa^r] * k « 4°"'£ L4 Jm 

1 * 0 


J 

V 


where j <s ’l 


is the Gamma function ( o{ ) 


v° 


Combining terms and simplifying 




Equivalently 


I [a -rU-T] 


0 ( 


y V 

3 J 
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L 


o( + F> 


ly - 1 


o )i 


(A - E, COf f, 0 I 0 



Where 


r <x +- Y x 

\ ~ 



K, 


+ t, 


of-v F, 



E^-E m a)) 2 ] 

Where 


(A-rJM? (LoiOJi 



(A 

& + 2 1 ^ 
V' I 




This process may be continued in the above manner. The details 
involve a change of variable for integration and substitution of the 
appropriate constants. An induction argument yields Equation 25 of 
the text. 
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APPENDIX D 


COMPARISON OF DISCRETE BAYESIAN TO THE CONTINUOUS 
BAYESIAN FAILURE RATE ESTIMATION 

1 # Introduction 

This comparison was prompted by the methods and formulations 
contined in Reference 8. The Bayesian formulation presented 
in this reference quantifies prior information discretely. The 
complete Bayesian formulation is developed via an assumed Weibull 
hazard- rate function. The discrete Bayesian formulation takes the 
following form: 

P(8 | A); PC4); 

^ P(B|A) ; P(A); •••,'» 

i - I 

where i = a particular cell 
n = number of cells 

P(A)^ = probability of event A (prior for celL) 

P (B 1 A) ^ = probability of event B given A for celL 
P (A|B)j^ = probability of event A given B for celL 

A three parameter Weibull assumption yields the following 
form for P (B|A): 


P( A | B ) 
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Where Q = 
M = 
E = 
R = 


Scale parameter (i.e.; failure rate = ) 

Shape parameter 
Position parameter 

Number of failures in the time interval 


II . Experiment 

Development of a discrete form of a Bayesian technique provided 
a significant departure from the continuous technique treated in the 
text. It was felt that a comparison experiment should be conducted. 
This experiment was designed to simultaneously compare three quantities 
the classical failure-rate-estimator and the expected values (means) 
for both the continuous and discrete Bayesian formulations. 

The standard exponential assumption is a special case of the 
Weilbull. Therefore, the appropriate parameter substitutions for the 
Weibull yielded the exponential function. The discrete computation 
algorithm was developed for an arbitrary number of failure (m) and sum 
of failure times (Zt). The posteriori probability for a given cell 
( \ -.1, t ) is given by the following equation: 

p 

iB\ = _ 

rm 

'N 


H 

The method of mathematical expectation (section 4.1) was used to 
derive the failure-rate values used for comparison. Discrete and 
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continuous prior distributions were made equivalent by equating 
the first and second moments of the respective priors (section 5). 
Computer-simulated times-to-failure were used as the comparative 
data base. Several prior variations were then considered. The 
expected value comparison was then conducted at each of thirty 
simulated data points. 

III. Results 

The results of this comparative simulation show no substantial 
difference between the expected values of the respective continuous 
and discrete Bayesian forms. This implies that the results reported 
in section 6 also hold for the discrete Bayesian when compared with 
the classical estimation technique. There are more subtle ways in 
which the continuous and discrete Bayesian forms differ. These 
differences are summarized below. 

A. Discrete Bayesian Advantages over Continuous Gamma 

1. Prior and subsequent distributions can represent multi- 
modal situations. 

2. Formulation of the prior distribution with respect 
to limited information may be more compatible with 
"engineering judgment . " 

3. The discrete Bayesian form appears to be the only way in 
which the increasing failure rate can be treated. 

i / 

4 . Complexity due to selection of mean and variance values 
and related distribution constants is eliminated. 
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B. Discrete Bayesian Disadvantages 


1. The posteriori of the discrete Bayesian can never 
achieve a value not given by the prior distibution. 
Therefore, if the prior is in error (i.e. the prior density 
is not positive at the value, the discrete Bayesian will not 
converge to the true value with the incorporation of data. 

2. The refinement of the discrete posteriori estimation process 
is dependent upon the cell size. 
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